Perplexity is the normalized inverse probability of the test set. It combines the "surprise" (inverse probability) of each word and then finds the geometric mean to get a per-word score.